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COMPARISON  OF  ALPHANUMERIC  DATA  ENTRY  METHODS 


FOR  ADVANCED  HELICOPTER  COCKPITS 


BACKGROUND 

The  visual  and  manual  workload  of  U.S.  Army  helicopter  crews  is  extremely 
high.  Flying  while  concurrently  navigating,  monitoring  aircraft  subsystems, 
and  performing  gunnery  or  reconnaissance  tasks  at  altitudes  below  treetop 
level  often  requires  more  time  and  attention  than  is  available.  The  U.S.  Army 
is  currently  formulating  concepts  for  a  family  of  light  helicopters  (LHX)  with 
automation  levels  that  may  allow  these  tasks  to  be  performed  by  a  single 
pilot.  Obviously,  computer  technology  is  required  to  automate  the  cockpit. 
At  the  present  time,  keypads  or  keyboards  are  required  for  entering 
alphanumeric  data  into  computerized  systems.  There  is  concern  that  the  visual 
and  manual  demands  of  operating  a  keyboard  in  flight  will  contribute  to  an 
even  greater  workload.  In  this  study,  two  alternatives  to  keyboards  are 
considered:  automatic  voice  recognition  and  data  entry  through  a  single, 
thumb-controlled  switch. 

Previous  research  has  been  performed  comparing  voice  and  keyboard  data 
entry  methods  in  airborne  applications  (Aretz,  1983;  Wyatt,  1983;  Malkin  & 
Christ,  1985a).  These  studies  indicate  that  data  entry  tends  to  be  faster  by 
keyboard  than  by  voice.  However,  the  aviators  who  participated  preferred 
voice  data  entry  because  it  was  perceived  as  requiring  less  effort.  The 
aviators  in  the  Aretz  study  qualified  their  preference  by  stating  that  when 
data  entry  is  the  sole  task  (not  performed  concurrently  with  flying)  or  when 
data  input  strings  are  short,  the  keyboard  is  the  preferred  method. 

These  three  studies  used  isolated-word  voice  recognition  systems  that 
require  a  l-to-2-second  pause  between  each  utterance.  A  connected-word  voice 
recognitioi.  system  is  used  in  this  study.  The  connected-word  voice 
recognition  system  accepts  a  string  of  predefined  utterances  without  pauses. 

A  pilot  study  was  conducted  to  determine  if  the  connected-word  voice 
recognizer  would  decrease  the  time  to  input  alphanumeric  data  strings  enough 
to  compete  with  the  keyboard  method.  Subjects  entered  the  data  strings  on  an 
isolated-word  voice  recognition  system  and  a  connected-word  voice  recognition 
system.  The  isolated-word  system  required  the  subjects  to  pause  between  each 
utterance.  On  the  connected-word  system,  pauses  were  not  permitted.  If  the 
subject  paused  in  the  middle  of  the  string,  the  recognizer  did  not  accept  the 
input.  The  subject  then  had  to  begin  again.  The  times  obtained  from  this 
study  showed  that  the  connected-word  system  was  significantly  faster  than  the 
isolated-word  system.  There  were  also  fewer  errors  with  the  connected-word 
system. 

Automatic  voice  recognition  appears  to  have  potential  as  an  alternative 
to  keyboard  data  entry,  but  the  evidence  to  date  suggests  that  speech 
technology  has  not  yet  fully  matured  for  use  in  the  high-stress  environment  of 
the  helicopter  cockpit  (Malkin  &  Christ,  1985b). 
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As  an  alternate  manual  method  to  the  keyboard,  the  Aviation  and  Air 
Defense  Division,  Human  Engineering  Laboratory  (HEL),  Aberdeen  Proving  Ground, 
Maryland,  has  developed  a  data  entry  technique  that  uses  a  thumb-controlled 
switch  to  scroll  alphanumeric  characters  onto  a  video  display.  The  hardware 
and  software  for  this  data  entry  technique  have  been  installed  in  an  HEL 
advanced  helicopter  flight  simulator  known  as  the  Cockpit  Research, 
Experimentation,  and  Workload  (CREW)  Simulator. 


OBJECTIVE 

The  objective  of  this  investigation  was  to  conduct  a  laboratory  flight 
simulation  to  compare  a  cockpit  keyboard,  a  thumb-controlled  switch,  and  a 
connected-word  voice  recognizer  for  data  entry  of  navigation  map  coordinate 
sets.  The  data  entry  methods  were  compared  under  two  task  conditions:  (a) 
the  entry  of  universal  transverse  mercator  (UTM)  coordinate  sets  as  the  sole 
task  performed  and  (b)  the  entry  of  UTM  coordinate  sets  performed  concurrently 
with  controlling  a  helicopter  simulator  while  flying  a  computer-generated 
external  scene.  The  time  and  error  differences  among  the  three  methods  of 
data  entry  were  evaluated  along  with  flight  performance  data  to  determine  the 
merits  of  each  method.  In  addition,  a  subjective  questionnaire  was  used  to 
determine  if  a  user  preference  exists. 


METHODOLOGY 

Subjects 

Data  were  collected  using  12  Army  aviators  assigned  to  Aberdeen  Proving 
Ground  (APG).  The  subjects  had  an  average  of  2,580  flight  hours  in  rotary- 
wing  aircraft,  with  a  range  of  1,100  to  5,000  hours.  The  average  age  was  38, 
with  a  range  of  32  to  50.  Nine  subjects  had  previous  experience  using  voice 
recognition  systems  as  a  result  of  participating  in  previous  investigations. 
Six  reported  having  previous  experience  with  keyboards,  but  none  had  previous 
experience  with  the  keyboard  arrangement  used  in  this  investigation.  None  had 
previous  experience  with  the  thumb-controlled  switch. 


Apparatus 

The  following  apparatus  was  used: 

1.  Flight  simulator  -  The  CREW  Simulator  consists  of  a  cockpit  cab 
with  advanced  controls  and  displays  and  an  "out-the-window"  scene  produced  by 
computer-generated  imagery  (CGI)  (Figure  1).  The  CGI  system  models  a  gaming 
area  of  5  square  miles  consisting  of  trees,  hills,  rivers,  roads,  and 
buildings.  A  40-by  40-degree  field  of  view  is  projected  on  a  6-foot  by  6-foot 
screen  in  the  front  of  the  cockpit  cab.  The  cockpit  controls  and  displays  are 
driven  by  a  VAX  11/750 ®  computer .  A  VAX  11/780®  computer  is  used  for  the  CGI 
and  experimental  data  collection.  Clark  and  Pferdner  (1985)  provide  more 
detailed  information  concerning  the  CGI  system. 
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Figure  1.  Cockpit  Research,  Experimentation,  and  Workload  (CREW)  Simulator. 


2.  Plasma  display  -  The  display  provides  data  entry  feedback  for 
each  data  entry  method. 

3.  Keyboard  -  The  keyboard  layout,  which  is  one  recommended  by 
Butterbaugh  and  Rockwell  (1982),  has  separate  keys  for  each  alpha  and  each 
numeric  character.  The  keyboard  and  plasma  display  are  located  in  the 
simulator  cockpit  on  the  left  console  adjacent  to  the  collective  lever  flight 
control  (Figures  2  and  3). 

4.  Thumb-controlled  switch  -  The  thumb-controlled  switch  is  a 
"Chinese  hat-type"  switch  located  on  top  of  a  standard  cyclic  flight  control 
handgrip  (Figure  4).  The  switch  has  four  positions  —  up,  down,  left,  and 
right.  The  plasma  display  is  positioned  on  the  upper  center  portion  of  the 
instrument  panel  (Figure  5).  The  intent  was  to  provide  the  pilot  with  the 
capability  of  entering  data  while  keeping  his  hands  on  the  flight  controls  and 
his  head  up. 


5.  Connected-word  voice  recognizer  -  The  Verbex  ®  Series  4000 
connected-word  voice  recognition  system  was  used.  It  makes  the  entry  of 
predefined  strings  of  words  without  pauses  between  the  words  possible.  Data 
entry  feedback  was  provided  on  the  plasma  display  in  the  same  location  as  the 
thumb-controlled  switch. 

6.  DECtalk  ®  voice  synthesizer  -  The  voice  synthesizer  provides 
seven  different  computer-generated  voices.  The  DECtalk®  default  voice,  a 
typical  male  voice,  was  used  during  the  study  to  prompt  the  subjects. 

7 .  Voice-activated  switch  -  The  switch  signaled  the  computer  when 
the  subject  began  entering  data  by  voice.  The  computer  then  began  collecting 
data . 


8.  Video  tape  recorder  -  This  was  used  for  data  collection. 


Procedure 

Training  and  testing  of  one  data  entry  method  was  completed  before  the 
next  data  entry  method  was  presented.  The  presentation  order  of  the  data 
entry  methods  and  the  flight  tasks  was  counterbalanced. 

Each  subject  was  given  an  overview  of  the  experiment  describing  the 
purpose  of  the  study  and  the  procedures  to  be  followed.  They  were  permitted 
to  ask  questions  concerning  the  investigation  and  their  involvement  in  the 
study.  Upon  completion  of  the  overview,  a  background  questionnaire  was 
administered  (see  Appendix  A). 


Training 

Regardless  of  the  order  in  which  the  data  entry  methods  were  tested, 
template  enrollment  for  the  voice  recognition  system  occurred  first.  The 
format  of  the  data  to  be  entered,  as  well  as  the  correction  procedures 
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Figure  2.  Keyboard  layout. 
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Figure  4.  Thumb-controlled  switch. 
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Figure  5.  Switch  and  display  location. 


available,  was  outlined  and  explained.  The  vocabulary  for  this  study 
consisted  of  20  words.  The  vocabulary  and  grammar  structure  are  included  in 
Appendix  B. 

Template  enrollment  involves  repeating  each  vocabulary  word  twice, 
followed  by  speaking  character  strings  formulated  randomly  by  the  recognition 
system  to  form  voice  templates.  Speaking  the  character  strings  is  referred  to 
as  embedded  training  and  is  required  for  operation  of  a  connected-word  voice 
recognizer.  It  provides  the  recognizer  with  samples  of  how  the  subject 
pronounces  certain  words  in  the  middle  of  a  string  of  words. 

Two  training  passes  were  used  to  form  the  subject’s  voice  templates. 
During  the  training  session,  the  subject  was  seated  in  the  simulator  and  wore 
his  flight  helmet  with  a  standard  aviation  microphone.  The  word  or  words  to 
be  spoken  appeared  on  a  cathode-ray  tube  (CRT)  located  on  the  left  side  of  the 
simulator.  The  subject  controlled  the  pace  at  which  the  words  were  presented. 
The  enrollment  process  took  approximately  30  minutes  to  complete. 

After  the  initial  voice  enrollment  was  completed,  a  recognition  test  was 
performed  to  determine  if  any  individual  words  required  retraining.  The 
subject  spoke  each  vocabulary  word  and  several  strings  of  words  two  times. 
Any  words  that  were  not  correctly  recognized  both  times  were  retrained. 

Once  the  voice  enrollment  session  was  complete,  the  subject  received 
training  on  the  flight  simulator  used  in  this  study.  The  flight  display  and 
the  unique  flight  characteristics  of  the  simulator  were  explained  to  the 
subject.  Two  practice  flight  sessions,  each  about  30  minutes  long,  were 
provided  to  familiarize  the  subject  with  the  simulator  and  the  predefined 
flight  course. 

One  potential  subject  was  precluded  from  participating  in  the  study 
because  he  was  unable  to  adapt  to  the  flight-handling  qualities  of  the 
simulator . 

Before  the  voice  data  entry  method  was  tested,  subjects  practiced 
entering  16  waypoint  coordinate  sets.  A  coordinate  set  consists  of  two  alpha 
characters  followed  by  eight  digits  (AB12345678) .  The  subjects  could  enter  a 
coordinate  set  one  character  at  a  time  or  any  combination  up  to  all  10 
characters  at  one  time.  The  inputs  were  displayed  on  the  plasma  display 
located  on  the  front  console  of  the  simulator.  The  error  correction 
procedures  were  also  practiced  to  ensure  that  the  subject  understood  the 
correction  process  of  each  method. 

Two  error  correction  procedures  were  available  to  correct  any  error.  The 
word  CLEAR  could  be  spoken  to  erase  the  display.  All  characters  entered  up  to 
that  point  would  be  erased.  The  subject  would  then  re-enter  the  coordinate 
set  from  the  beginning.  The  second  option  was  taken  from  a  study  performed  at 
the  Naval  Postgraduate  School  (Poock  and  Martin,  1985).  The  string  "POSITION 
(digit)  MAKZ-IT  (alphanumeric)"  had  to  be  spoken  all  at  once  without  any  long 
pauses  between  words.  The  "digit"  could  be  a  number  1  through  10 
corresponding  to  the  position  of  the  error  in  the  coordinate  set.  Above  each 
position  on  the  plasma  display,  the  number  corresponding  to  that  position  was 
indicated.  The  "alphanumeric"  referred  to  the  correct  number  or  alpha 
character  to  be  input.  For  example,  if  it  was  necessary  in  the  coordinate  set 
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AB12945678  to  change  the  digit  9  to  the  digit  3,  the  command  to  be  spoken 
would  be  "POSITION  FIVE  MAKE-IT  THREE.”  Once  the  proper  coordinate  set  was 
displayed,  the  word  ENTER  was  spoken  to  enter  the  coordinate. 

A  subject  was  considered  proficient  if,  after  the  practice  session,  five 
coordinate  sets  were  entered  correctly.  A  second  subject  had  to  be  replaced 
due  to  poor  performance  on  the  voice  recognition  system.  After  three  training 
passes  on  the  entire  vocabulary,  along  with  additional  retraining  on  the 
individual  words  ENTER,  EIGHT,  and  CORRECTION,  a  coordinate  set  could  not  be 
correctly  entered.  The  subject  had  prior  experience  with  using  an  isolated- 
word  voice  recognition  system  without  any  difficulty. 

Training  on  the  keyboard  method  of  data  entry  consisted  of  an  explanation 
of  data  entry  and  the  error  correction  procedures,  followed  by  a  practice 
session.  The  keyboard  used  had  separate  keys  for  the  alpha  and  numeric 
characters.  Therefore,  only  one  key  press  was  required  to  enter  any 
character . 

Two  error  correction  procedures  were  available  for  the  keyboard  data 
entry  method.  The  subject  could  depress  the  CLEAR  key,  in  which  case  the  data 
entered  up  to  that  point  would  be  erased.  The  correct  coordinate  set  could 
then  be  entered  from  the  beginning.  The  other  correction  procedure  was  the 
use  of  the  backward  and  forward  arrow  keys.  Each  time  one  of  the  keys  was 
depressed,  the  cursor  would  move  one  space  forward  or  backward.  The  subject 
could  then  enter  the  correct  characters,  starting  at  the  position  of  the 
cursor.  Once  the  correct  coordinate  set  was  displayed,  the  ENTER  key  was 
depressed  to  enter  the  coordinate  into  the  system. 

Subjects  were  able  to  practice  entering  and  correcting  16  coordinate  sets 
using  the  keyboard.  The  characters  were  displayed  as  they  were  entered  on  the 
plasma  display,  located  adjacent  to  the  keyboard.  The  same  proficiency 
criterion  used  for  voice  data  entry  was  also  used  for  keyboard  data  entry. 

The  thumb-controlled  switch  located  on  the  cyclic  was  the  third  method  of 
data  entry.  The  switch  had  four  positions:  left,  right,  up,  and  down.  The  up 
and  down  positions  controlled  the  forward  and  backward  scrolling  of  the 
alphanumeric  characters;  the  left  and  right  positions  controlled  the  position 
of  the  cursor.  To  enter  a  character,  the  subject  held  the  switch  either  up  or 
down,  scrolled  through  the  alphabet  or  numbers  until  the  desired  one  to  be 
entered  was  displayed,  and  then  moved  the  cursor  to  the  next  position  in  the 
coordinate  set.  Only  alpha  characters  were  displayed  in  the  first  two 
positions  and  only  digits,  in  the  last  eight.  The  characters  were  displayed 
on  the  plasma  display  located  on  the  front  console  of  the  simulator. 

Error  correction  was  handled  by  positioning  the  cursor  under  the 
character  to  be  corrected  by  using  the  left  or  right  position  on  the  switch, 
then  rescrolling  through  the  alphabet  or  numbers  until  the  correct  entry  was 
displayed.  To  enter  the  correct  coordinate  set,  the  switch  was  pushed  to  the 
right. 

Before  each  method  of  data  entry  was  tested,  each  subject  was  given  a 
practice  flight  in  the  simulator,  during  which  time  four  coordinate  sets  were 
entered.  This  gave  the  subjects  a  feel  for  how  the  test  session  would  be 
conducted . 


Testing 


There  were  two  testing  conditions:  a  flight  condition  and  a  no-flight 
condition.  Each  subject  entered  eight  UTM  coordinate  sets  for  each  test 
condition.  The  coordinate  sets,  which  were  selected  from  a  military  map  of 
the  Fulda  Gap  area  of  Germany,  were  located  on  a  kneeboard  attached  to  the 
subject's  leg.  A  different  set  of  coordinates  was  used  in  each  condition. 
The  subject  was  tested  in  both  conditions  using  one  data  entry  method  before 
proceeding  to  the  next  data  entry  method.  Flight  gloves  were  worn  by  the 
subjects  during  all  training  and  testing  conditions. 

In  the  flight  condition,  each  subject  entered  the  coordinate  sets  while 
flying  the  predefined  course  in  the  simulator.  The  flight  course  was  along  a 
winding  riverbed  lined  on  both  sides  by  trees  and  hills.  Each  subject  was 
instructed  to  fly  the  course  as  fast  as  possible  while  remaining  at  an 
altitude  at  or  below  treetop  level  and  centered  on  the  riverbed.  Directional 
poles  were  positioned  at  river  and  road  intersections  to  assist  the  subject  in 
staying  on  course.  The  DECtalk  ® voice  synthesizer  gave  additional  directional 
guidance  to  the  subject . 

The  computer  monitored  progress  along  the  course,  and  at  predefined 
locations  the  subjects  were  prompted  to  enter  each  coordinate  set.  The  prompt 
was  given  by  the  voice  synthesizer.  The  subjects  were  instructed  to  enter 
data  as  they  would  in  an  actual  flight;  that  is,  as  quickly  and  as  accurately 
as  possible  while  maintaining  safe  operation  of  the  aircraft. 

In  the  no-flight  condition,  data  entry  was  performed  as  a  sole  task.  The 
subject  was  seated  in  the  simulator  cockpit  and  entered  the  coordinate  sets. 
The  voice  synthesizer  prompted  the  subject  when  to  enter  each  coordinate.  A 
random  time  between  5  to  12  seconds  was  allowed  between  each  waypoint.  These 
delay  times  were  the  same  for  each  subject. 

When  all  testing  was  completed,  each  subject  was  given  a  subjective 
questionnaire  to  complete  (see  Appendix  C) . 


Data  Collection 

All  data,  except  for  the  voice  data  entry  errors,  were  collected  by  the 
computer.  The  data  collected  were  as  follows: 

1.  Response  time  -  Response  time  is  the  time  from  when  the  subject 
was  prompted  to  when  data  entry  began.  When  voice  data  entry  was  being 
tested,  the  prompt  to  enter  data  also  turned  on  the  voice-activated  switch. 

2.  Input  time  -  Input  time  is  the  time  from  the  start  of  data  entry 
to  the  completion  of  data  entry.  When  voice  data  entry  was  being  tested,  the 
voice-activated  switch  was  used  to  record  the  inputs  made  by  the  subjects. 
When  the  other  two  methods  were  being  tested,  the  character  inputs  indicated 
the  input  time. 
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3.  Total  time  -  Total  time  la  the  time  from  when  the  subject  was 
prompted  to  enter  data  to  the  completion  of  data  entry.  It  was  recorded  by 
adding  the  response  time  and  the  Input  time. 

4.  Errors  -  Voice  data  entry  errors  were  scored  using  the  best-case 
pattern  matching  procedure  as  outlined  by  the  National  Bureau  of  Standards 
(Pallett,  1985).  The  errors  consisted  of  the  following: 

a.  Nonrecognition  -  The  recognizer  did  not  accept  the  Input. 

b.  Misrecognition  -  The  recognizer  misunderstood  the  Input. 

c.  Insertion  -  The  recognizer  inserted  a  character  into  the 
string  put  in  by  a  speaker. 

d.  Deletion  -  The  recognizer  dropped  a  character  from  the 

input  string. 

e.  Subject  error  -  The  speaker  inputted  the  wrong  character. 

f.  Error  of  technique  -The  speaker  inserted  lengthy  pauses 
into  the  character  string. 

These  errors  were  collected  off  the  video  tape  recordings  made  during  the  test 
sessions.  Keyboard  and  thumb-controlled  switch  errors  consisted  only  of 
subject  errors.  The  computer  compared  the  individual  character  entries  made 
by  the  subject  to  the  correct  character  that  should  have  been  entered.  Errors 
were  then  recorded. 

5.  Flight  performance  -  Flight  performance  data  were  sampled  four 
times  per  second,  averaged,  and  printed  once  every  second.  The  flight 
performance  data  recorded  were  airspeed,  altitude,  and  flight  path  deviation. 
Flight  path  deviation  was  measured  as  the  root-mean-square  (RMS)  error  from 
the  set  path.  The  differences  between  "before  data  entry”  and  "during  data 
entry"  for  airspeed,  altitude,  and  flight  path  deviation  were  compared. 


Experimental  Design 

The  data  collected  were  broken  down  into  two  groups  to  be  analyzed 
separately.  In  Design  1,  a  2  x  3  x  12  factorial  design  with  repeated  measures 
on  the  12  subjects  was  used.  The  within-sub ject  factors  were  data  entry 
methods  (voice,  keyboard,  and  thumb-controlled  Bwitch)  and  task  conditions 
(flight,  no  flight).  The  dependent  variables  were  response  time,  input  time, 
and  errors. 

A  3  x  12  factorial  design  with  repeated  measures  on  the  subjects  was  used 
for  the  second  design.  The  within-subject  factor  was  data  entry  methods 
(voice,  keyboard,  and  thumb-controlled  switch).  The  dependent  variables  were 
the  average  differences  between  before  data  entry  and  during  data  entry  for 
airspeed,  altitude,  and  flight  path  deviation. 

The  presentation  order  of  the  data  entry  methods,  as  well  as  the  task 
conditions,  was  counterbalanced. 


RESULTS 


The  data  were  analyzed  uBing  the  Statistical  Package  of  the  Social 
Sciences  (SPSS)  User's  Guide  (SPSS,  1986).  In  Design  1,  subjects  were 
measured  across  the  various  treatment  combinations  for  each  dependent 
variable.  Because  the  measures  were  highly  correlated,  a  multivariate 
analysis  was  performed  using  the  univariate  repeated  measures  model  with 
subjects  considered  a  random  factor.  Summary  analysis  tables  for  the  measures 
analyzed  in  this  design  are  in  Appendix  D.  As  a  confirmatory  procedure,  the 
results  achieved  using  this  procedure  were  compared  with  the  results  obtained 
using  a  multivariate  repeated  measures  model.  The  results  agreed. 

Design  2  was  analyzed  similarly  to  Design  1 .  Subjects  were  measured 
across  data  entry  method  and  task  condition  for  the  flight  performance 
measures.  A  multivariate  analysis  was  again  performed  using  the  univariate 
repeated  measures  model  with  subjects  considered  a  random  factor. 

Confidence  intervals  (95th  percentile)  were  calculated  to  test  for 
differences  between  the  means  for  each  of  the  significant  dependent  measures. 
Results  of  the  analyses  for  the  various  dependent  measures  are  described  in 
the  following  sections.  The  mean,  standard  deviation,  and  confidence  interval 
tables  are  in  Appendix  E. 


Response  Time 

Subjects  were  able  to  respond  significantly  faster  to  the  prompt  to  enter 
data  during  the  no-flight  task  condition  than  during  the  flight  task 
condition,  £(  1 ,  11)  •  29.25,  £  <  .05.  There  also  was  a  significant 

interaction  between  data  entry  method  and  task  condition,  £(2,  22)  -  11.7,  £  < 
.05.  During  the  no-flight  task  condition,  subjects  responded  significantly 
faster  to  the  prompt  when  the  keyboard  was  used  to  enter  data  as  opposed  to 
voice  or  the  thumb-controlled  switch.  However  during  the  flight  task 
condition,  subjects  responded  significantly  faster  using  either  voice  or  the 
thumb-controlled  switch  to  enter  data  rather  than  with  the  keyboard  method 
(see  Figure  6). 


Input  Time 

There  was  a  significant  difference  between  each  data  entry  method  for  the 
mean  input  time,  £(2,  22)  “  30.78,  £  <  .05.  The  keyboard  method  was  the 
fastest  mode,  followed  by  the  voice  data  entry  method  and  then  by  the  thumb- 
controlled  switch  method,  under  each  task  condition.  Subjects  were  able  to 
input  data  significantly  faster  during  the  no-flight  task  condition  than 
during  the  flight  task  condition,  £(1,  11)  “  25.21,  £  <  .05  (see  Figure  7). 


Total  Time 

A  significant  difference  existed  between  the  mean  total  times  to  complete 
data  entry  using  each  method,  £(2,  22)  »  34.26,  £  <  .05.  Subjects  were  able 
to  respond  and  enter  data  faster  during  the  no-flight  task  condition  than 
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Figure  6.  Mean  response  time. 
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Figure  7.  Mean  input  time. 
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during  Che  flight  task  condition,  _F(1,  11)  =  31.17,  £  <  .05.  During  the  no¬ 
flight  task  condition,  the  total  time  to  enter  data  was  significantly  faster 
using  the  keyboard  method  than  either  the  voice  or  thumb-controlled  switch 
methods.  During  the  flight  task  condition,  the  voice  data  entry  and  keyboard 
methods  were  significantly  faster  for  overall  total  time  than  the  thumb- 
controlled  switch  method  (see  Figure  8). 


Errors 

There  were  significantly  more  errors  with  voice  data  entry  than  with  the 
keyboard  or  the  thumb-controlled  switch;  £(2,  22)  =  20.30,  £  <  .05  (see  Figure 
9).  The  majority  of  the  errors  that  were  made  using  voice  data  entry  are 
attributed  to  machine  errors,  not  subject  errors.  When  only  subject  errors 
were  compared,  no  significant  differences  existed.  An  overall  recognition 
accuracy  of  81  percent  was  attained  in  this  study.  The  recognition  system 
correctly  recognized  85.5  percent  of  the  utterances  when  data  entry  was  a  sole 
task  and  75.9  percent  during  the  flight  task  condition.  Recognition  rates  for 
the  12  subjects  ranged  from  58  percent  to  99  percent.  The  ENTER  command  was 
consistently  nonrecognized  across  subjects.  In  one  case,  it  was  misrecognized 
as  the  command  CLEAR. 


Airspeed 

There  were  no  significant  differences  for  airspeed.  While  entering  data 
using  each  of  the  three  data  entry  methods,  the  subjects  were  able  to  maintain 
close  to  the  same  airspeed  as  they  did  before  entering  data.  Table  1  shows 
the  average  difference  In  airspeed  between  flying  before  entering  data  and 
flying  while  entering  data.  The  difference  was  computed  by  subtracting  the 
average  airspeed  flown  before  entering  data  from  the  average  airspeed  flown 
while  entering  data.  Therefore,  the  negative  numbers  in  Table  1  indicate  that 
a  slightly  higher  airspeed  was  flown  before  data  entry  than  during  data  entry 
for  all  three  data  entry  methods. 


Altitude 

There  was  no  significant  difference  between  data  entry  methods  for 
altitude.  This  would  indicate  that  the  subjects  were  unaffected  by  the  method 
of  data  entry  in  their  ability  to  maintain  a  consistent  altitude.  A 
significant  difference  did  exist  between  subjects  for  the  average  altitude 
difference;  F(2,  22)  »  1.84,  £  <  .05.  The  difference  between  subjects 
indicates  that  some  subjects  did  not  maintain  the  same  altitude  while  entering 
data  as  they  did  before  entering  data.  However,  the  method  of  data  entry  used 
had  no  bearing  on  the  subjects’  abilities  to  maintain  a  consistent  altitude. 
The  average  differences  in  altitude  are  also  shown  in  Table  1.  The  positive 
values  indicate  that  a  slightly  higher  altitude  was  flown  during  data  entry 
than  before  data  entry. 
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Figure  8.  Mean  total  time. 
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Figure  9.  Mean  numbers  of  errors. 
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Table  1 


Data  Entry  Method 

By  Mean  Flight 

Performance 

Difference 

Voice 

Keyboard 

Thumb  switch 

Airspeed  (knots) 

-1.13 

-3.39 

-0.64 

Altitude  (feet) 

5.62 

1 .83 

8.33 

Flight  path 
deviation  -  RMS 
error  (feet) 

2.20 

19.20 

51.30 

Flight  Path  Deviation 

There  were  no  significant  differences  for  flight  path  deviation.  The 
subjects  were  able  to  maintain  their  flight  path  over  the  riverbed  for  each 
data  entry  method.  Table  1  also  contains  the  average  differences  in  flight 
path  deviation. 


Subjective  Data 

The  majority  of  the  pilots  preferred  to  enter  the  navigational  data  by 
voice  under  both  task  conditions.  The  thumb-controlled  switch  method  was  the 
least  preferred  method.  All  three  methods  were  viewed  as  easy  to  learn  to 
use.  The  results  are  shown  in  Table  2. 


DISCUSSION 

The  objective  of  this  study  was  to  compare  the  three  data  entry  methods 
for  time  and  errors  when  data  entry  was  the  sole  task  and  for  time,  errors, 
and  the  effect  on  flight  performance  when  data  entry  was  performed  while 
flying  a  helicopter  simulator.  As  the  results  show,  each  method  had  its 
relative  merits. 

For  response  time,  subjects  were  able  to  respond  faster  to  the  prompt  to 
enter  data  by  keyboard  when  data  entry  was  the  sole  task  performed.  The  list 
of  coordinate  sets  was  located  on  a  kneeboard  positioned  on  the  left  leg 
beside  the  keyboard.  The  subjects  rested  their  hands  on  the  keyboard,  waiting 
for  the  next  prompt  to  occur.  The  other  two  methods  required  the  subjects  to 
glance  down  at  the  coordinate  list  and  then  look  up  to  enter  and  check  the 
data . 
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Table  2 


Subjective  Questionnaire  Results  (%) 


Voice 

Keyboard 

Thumb  switch 

Prefer  most 
(No  flight) 

70 

15 

15 

Prefer  least 
(No  flight) 

15 

31 

54 

Prefer  most 
(Flight) 

67 

16.5 

16.5 

Prefer  least 
(Flight) 

16 

42 

42 

Easiest  to  learn 

44 

38 

18 

Most  difficult 
to  learn 

36 

28 

36 

These  results  were  reversed  when  data  entry  was  performed  while  flying 
the  simulator.  The  subjects  were  able  to  respond  to  the  prompt  to  enter  data 
faster  when  using  the  voice  or  thumb-controlled  switch  methods  than  with  the 
keyboard.  The  keyboard  method  required  the  subjects  to  remove  their  hands 
from  the  collective  and  enter  data  while  trying  to  fly  through  a  simulated 
low-level  flight  along  a  winding  riverbed.  If  the  prompt  to  enter  data  came 
at  a  time  in  the  flight  path  that  required  input  from  the  collective,  the 
subjects  could  not  enter  the  data  immediately.  The  voice  and  thumb-controlled 
switch  methods  did  not  require  the  subjects  to  remove  their  hands  from  the 
flight  controls.  The  subjects  could  continue  to  make  the  necessary  flight 
inputs  while  responding  to  the  prompt  to  enter  data. 

However,  once  data  entry  began,  the  keyboard  method  resulted  in  the 
fastest  entry  of  the  navigational  data.  Very  few  errors  were  committed  using 
the  keyboard.  The  majority  of  the  subjects  would  enter  the  two  alpha 
characters  first,  then  enter  the  digits  in  two  groups  of  four,  checking  each 
time  for  errors.  The  keyboard  was  also  a  more  reliable  system  compared  to  the 
voice  recognizer.  If  subjects  depressed  a  key,  they  were  certain  to  have  that 
character  displayed.  They  could  not  be  assured  that  the  character  they  spoke 
would  be  the  one  displayed. 
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The  thumb-controlled  switch  was  not  quite  as  reliable  as  the  keyboard 
either.  Some  confusion  existed  over  the  fact  that  the  clicks  that  occurred 
when  moving  the  switch  did  not  necessarily  correspond  to  the  scrolling  of  the 
characters.  The  switch  needed  to  be  pushed  past  the  location  where  the  click 
occurred . 

Voice  data  entry  waa  slowed  considerably  by  the  low  recognition  accuracy. 
In  a  diagnostic  probe  of  the  data,  the  input  time  was  separated  into  two 
groups:  error-free  input  time  and  input  time  with  errors.  No  significant 
difference  existed  for  error-free  input  time  between  the  keyboard  and  the 
voice  system  (15.8  seconds  versus  14.6  seconds,  respectively).  The  thumb- 
controlled  switch  method  was  still  significantly  slower  (39.3  seconds).  This 
would  seem  to  indicate  that  if  a  perfect  voice  recognition  system  was 
developed  that  could  correctly  accept  all  inputs  made  under  all  conditions  and 
by  all  speakers,  data  entry  by  voice  could  be  as  fast  as  the  current  keyboard 
method . 

Using  connected-word  voice  recognition  to  enter  long  strings  of 
alphanumerics  creates  an  error  detection-correction  problem.  As  with  the 
keyboard  method,  the  subjects  tended  to  enter  the  two  alpha  characters  first, 
followed  by  two  groups  of  four  digits  each.  An  individual  character  error 
could  not  be  detected  or  corrected  until  the  entire  spoken  string  was 
recognized.  The  procedure  was  very  time-consuming.  In  the  Instances  of 
insertions  and  deletions,  one  error  could  require  the  subject  to  go  through 
the  POSITION/MAKE-IT  error  correction  procedure  several  times  to  correct  one 
error.  At  the  very  least,  the  subject  could  clear  the  display  and  begin 
again,  which  was  also  time-consuming.  The  same  error  could  then  still  occur. 

The  total  time  required  to  enter  a  coordinate  set  with  the  keyboard  was 
significantly  faster  than  both  the  voice  and  thumb-controlled  switch  methods 
when  data  entry  was  a  sole  task.  However,  during  the  flight  task  condition, 
there  was  no  statistical  difference  between  the  voice  and  the  keyboard,  with 
the  thumb-controlled  switch  still  significantly  slower.  When  comparing  the 
mean  times  for  the  three  methods,  a  10-second  difference  existed  between  the 
keyboard  and  the  voice  system.  It  took  an  average  of  28.4  seconds  to  respond 
and  enter  data  by  the  keyboard  method  compared  to  38.7  seconds  using  the  voice 
data  entry  method. 

One  reason  there  was  no  statistical  difference  in  flight  performance 
measured  across  effects  could  be  due  to  the  flight  handling  qualities  of  the 
simulator.  The  other  reason  could  be  that  the  pilots  were  instructed  to  fly 
and  enter  data  as  they  would  in  an  operational  environment.  Thus  they  gave 
the  flight  task  their  primary  attention  and  entered  data  only  when  the 
aircraft  was  under  reasonable  control. 

Pilot  preference  for  the  method  of  data  entry  was  not  surprising.  Even 
with  the  high  error  rate,  the  majority  of  the  subjects  still  preferred  to 
enter  data  by  voice.  They  preferred  the  voice  method  because  it  allowed  them 
to  keep  their  attention  focused  outside  of  the  aircraft.  Although  the  thumb- 
controlled  switch  also  permitted  more  attention  focused  on  the  flight  rather 
than  on  the  data  entry,  the  majority  of  the  pilots  found  it  difficult  to 
operate  the  switch  without  moving  the  cyclic.  They  also  detected  that  the 
thumb-controlled  switch  was  considerably  slower  than  the  other  two  methods. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


Although  the  results  point  to  the  fact  that  entering  alphanumeric  data  by 
keyboard  is  faster  and  less  error-prone,  pilots  required  to  operate  such 
systems  prefer  to  use  voice  data  entry.  It  allows  pilots  to  devote  more 
attention  to  their  flight  task,  thereby  reducing  the  workload  associated  with 
time-sharing  between  two  or  more  tasks.  In  situations  such  as  low-level  or 
nap-of-the-earth  flight,  it  is  almost  impossible  for  a  pilot  to  divert  his 
attention  away  from  the  external  environment. 

However,  before  a  voice  recognition  system  can  be  used  in  such 
environments,  recognition  accuracy  needs  to  be  improved.  Perhaps  the  accuracy 
could  have  been  increased  by  limiting  the  syntax  structure  of  the  vocabulary. 
By  giving  the  pilots  as  much  flexibility  as  possible  in  the  number  of 
characters  that  could  be  entered  at  one  time,  recognition  accuracy  could  have 
been  adversely  affected. 

Overall,  based  on  the  results  of  this  study,  the  use  of  voice  recognition 
is  not  recommended  for  tasks  that  require  the  entry  of  relatively  long  digit 
or  alphanumeric  strings . 

An  area  of  connected-word  voice  recognition  that  needs  to  be  further 
Investigated  is  error  correction  procedures.  The  POSITION/MAKE-IT  method  used 
in  this  study  was  time-consuming.  Although  the  subjects  successfully  used 
this  method  during  practice  sessions,  they  sometimes  forgot  the  exact  wording 
of  the  command  during  the  test  sessions.  In  addition,  the  subjects  had  to 
determine  the  position  in  the  string  where  the  error  occurred,  adding  to  the 
time  involved  for  correcting  errors.  Alternative  methods,  perhaps  combining  a 
manual  device  with  the  voice  input,  may  reduce  the  time  required  to  correct 
errors . 
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BACKGROUND  QUESTIONNAIRE 


NAME:  _ 

AGE:  _ 

GRADE/RANK :  _ 

TOTAL  NO.  OF  YEARS  AS  A  RATED  AVIATOR:  _ 

TOTAL  HOURS  IN  ROTARY  WING:  _ 

TOTAL  HOURS  IN  FIXED  WING:  _ 

AVERAGE  NO.  OF  HOURS  YOU  ARE  CURRENTLY  FLYING  PER  MONTH:  _ 

DO  YOU  HAVE  ANY  EXPERIENCE  USING  SPEECH  RECOGNITION  EQUIPMENT? 
NONE  SOME  A  LOT 

WHEN  WAS  THE  LAST  TIME  YOU  USED  A  SPEECH  RECOGNIZER?  _ 

DO  YOU  HAVE  ANY  EXPERIENCE  USING  A  DATA  ENTRY  KEYBOARD? 

NONE  SOME  A  LOT 

WHEN  WAS  THE  LAST  TIME  YOU  USED  A  DATA  ENTRY  KEYBOARD? 
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APPENDIX  B 

VOCABULARY  AND  GRAMMAR  STRUCTURE 
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VOCABULARY 


ALPHA 

BRAVO 

MIRE 

NOVEMBER 

ZERO 

ONE 

TWO 

THREE 

FOUR 

FIVE 

SIX 

SEVEN 

EIGHT 

NINER 

TEN 

CORRECTION 
CLEAR 
ENTER 
POSITION 
MAKE- IT 


GRAMMAR  STRUCTURE 


•ALPHA  <*  2  .DIGIT  9  0,8 

•DIGIT  9  1,8 

POSITION  .PLACE  MAKE -IT  .ALPHANUM 

CORRECTION 

ENTER 

CLEAR 

.ALPHA  - 
ALPHA 
BRAVO 
MIKE 

NOVEMBER 

.DIGIT  - 
ONE 
TWO 
THREE 
FOUR 
FIVE 
SIX 
SEVEN 
EIGHT 
NINER 
ZERO 

.PLACE  - 

.DIGIT 

TEN 

•ALPHANUM  = 

.ALPHA 

.DIGIT 
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APPENDIX  C 

SUBJECTIVE  QUESTIONNAIRE 
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SUBJECTIVE  QUESTIONNAIRE 


1.  WHICH  DATA  ENTRY  METHOD  DO  YOU  PREFER  MOST  WHEN  DATA  ENTRY  IS  THE  SOLE 
TASK? 

WHICH  THE  LEAST? 

BRIEFLY  EXPLAIN  THE  REASONS  FOR  YOUR  PREFERENCES. 

ADDITIONAL  COMMENTS: 

2.  WHICH  DATA  ENTRY  METHOD  DO  YOU  PREFER  MOST  WHEN  DATA  ENTRY  IS  PERFORMED 
WHILE  FLYING? 

WHICH  THE  LEAST? 

BRIEFLY  EXPLAIN  THE  REASONS  FOR  YOUR  PREFERENCES. 

ADDITIONAL  COMMENTS: 

3.  IN  YOUR  EXPERIENCE  TODAY,  WHICH  DATA  ENTRY  METHOD  WAS  THE  EASIEST  TO  LEARN 
TO  USE? 

WHICH  WAS  MOST  DIFFICULT? 

BRIEFLY  EXPLAIN  THE  REASONS  FOR  YOUR  CHOICES. 

ADDITIONAL  COMMENTS: 
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APPENDIX  D 

MULTIVARIATE  ANALYSIS  OF  VARIANCE  (MANOVA)  RESULTS 
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MANOVA  RESULTS 


Statistical 

Significance  Of  Factors 

By  Dependent  Measures 

Dependent  Measures 

^-Statistics 

Subject 

Significance  a  =  .05 

Response  Time 

8.64 

* 

Input  Time 

5.57 

* 

Total  Time 

7.70 

* 

Total  Errors  (Log) 

2.78 

* 

Dependent  Measures 

F-Statistics 

Method 

Significance  a  =  .05 

Response  Time 

1.07 

* 

Input  Time 

30.78 

* 

Total  Time 

34.26 

* 

Total  Errors  (Log) 

20.30 

* 

Dependent  Measures 

F-Statistics 

Task 

Significance  a  *  .05 

Response  Time 

29.25 

* 

Input  Time 

20.21 

* 

Total  Time 

31.17 

* 

Total  Errors  (Log) 

3.41 

- 

Note .  *  denotes  statistical  significance. 

-  denotes  no  statistical  significance. 

(table  continues) 
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Dependent  Measures 

Method 

F-Statistics 

By  Task 

Significance  a  =  .05 

Response  Time 

11.7 

★ 

Input  Time 

2.25 

- 

Total  Time 

2.48 

- 

Total  Errors  (Log) 

2.84 

- 

Subject 

By  Method  By  Task 

Dependent  Measures 

^-Statistics 

Significance  a  =  .05 

Response  Time 

1 .13 

- 

Input  Time 

1 .00 

- 

Total  Time 

.97 

- 

Total  Errors  (Log) 

1.03 

- 

Subject 

By  Task 

Dependent  Measures 

F-Statistics 

Significance  a  =  .05 

Response  Time 

4.09 

* 

Input  Time 

2.45 

k 

Total  Time 

3.11 

k 

Total  Errors  (Log) 

1.50 

- 

Subject 

By  Method 

Dependent  Measures 

F-Statistics 

Significance  o  =  .05 

Response  Time 

3.15 

k 

Input  Time 

2.09 

k 

Total  Time 

2.26 

k 

Total  Errors  (Log) 

3.19 

k 
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MANOVA  RESULTS 


Statistical 

Significance  Of  Factors 

By  Dependent  Measures 

Dependent  Measures 

F-Statistics 

Subject 

Significance  a  = 

.05 

Speed  Difference 

1.57 

.107 

Altitude  Difference 

1.84 

.048 

Error  Difference 

1.26 

.247 

Dependent  Measures 

^-Statistics 

Method 

Significance  a  =. 

05 

Speed  Difference 

1.12 

.342 

Altitude  Difference 

2.98 

.071 

Error  Difference 

2.92 

.075 

Subject  By  Method 

Dependent  Measures 

F-Statistics 

Significance  a  = 

.05 

Speed  Difference 

1.11 

.332 

Altitude  Difference 

.72 

.812 

Error  Difference 

.81 

.70 
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APPENDIX  E 

MEAN,  STANDARD  DEVIATION,  AND  CONFIDENCE 
INTERVAL  TABLES 


45 


RESPONSE  TIME 


Voice 

Keyboard 

Thumb-controlled 


Voice 

Keyboard 

Thumb-controlled 


Voice 

Keyboard 

Thumb-controlled 


Voice 

Keyboard 

Thumb-controlled 


NO  FLIGHT 


switch 


switch 


switch 


switch 


Mean 

Standard 

Deviation 

95% 

Confidence 

Interval 

2.34 

1.26 

2.085 

2.597 

1.23 

.75 

1.085 

1.389 

1.97 

1.89 

1.591 

2.355 

Mean 

FLIGHT 

Standard 

Deviation 

95% 

Confidence 

Interval 

3.53 

3.32 

2.830 

4.239 

6.72 

6.65 

5.376 

8.070 

5.54 

7.06 

4.067 

7.008 

INPUT  TIME 

NO  FLIGHT 

Standard 

95% 

Mean 

Deviation 

Confidence 

Interval 

26.56 

21.77 

22.147 

30.970 

12.46 

4.77 

11.493 

13.426 

32.50 

7.28 

31.027 

33.977 

FLIGHT 

Standard 

95% 

Mean 

Deviation 

Ccnf idence 

Interval 

35.17 

34.80 

27.792 

42.541 

21.70 

11.02 

19.468 

23.933 

48.51 

19.45 

44 .464 

52.563 
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TOTAL  TIME 


NO  FLIGHT 


Mean 

Standard 

Deviation 

952 

Confidence 

Interval 

Voice 

28.90 

21.88 

24.466 

33.334 

Keyboard 

13.70 

4.86 

12.711 

14.681 

Thumb-controlled  switch 

34.47 

7.07 

33.042 

35.908 

FLIGHT 

Standard 

952 

Mean 

Deviation 

Confidence 

Interval 

Voice 

38.70 

35.76 

31.125 

46.277 

Keyboard 

28.42 

13.98 

25.592 

31.256 

Thumb-controlled  switch 

54.05 

21.63 

49.547 

58.555 

ERRORS  (LOG) 

NO  FLIGHT 

Standard 

952 

Mean  Deviation 

Confidence  Interval 

Voice 

2.48 

.28 

2.425 

2.537 

Keyboard 

2.32 

.07 

2.308 

2.338 

Thumb-controlled  switch 

2.33 

.07 

2.317 

2.346 

FLIGHT 

Standard 

952 

Mean 

Deviation 

Confidence 

Interval 

Voice 

2.57 

.39 

“  2.483 

2.648 

Keyboard 

2.33 

.08 

2.311 

2.342 

Thumb-controlled  switch 

2.35 

.08 

2.332 

2.367 
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